CMFAN: Cross-Modal Feature Alignment Network for Few-Shot Single-View 3D Reconstruction.

Few-shot single-view 3D reconstruction learns to reconstruct the novel category objects based on a query image and a few support shapes. However, since the query image and the support shapes are of different modalities, there is an inherent feature misalignment problem damaging the reconstruction. Previous works in the literature do not consider this problem. To this end, we propose the cross-modal feature alignment network (CMFAN) with two novel techniques. One is a strategy for model pretraining, namely, cross-modal contrastive learning (CMCL), here the 2D images and 3D shapes of the same objects compose the positives, and those from different objects form the negatives. With CMCL, the model learns to embed the 2D and 3D modalities of the same object into a tight area in the feature space and push away those from different objects, thus effectively aligning the global cross-modal features. The other is cross-modal feature fusion (CMFF), which further aligns and fuses the local features. Specifically, it first re-represents the local features with the cross-attention operation, making the local features share more information. Then, CMFF generates a descriptor for the support features and attaches it to each local feature vector of the query image with dense concatenation. Moreover, CMFF can be applied to multilevel local features and brings further advantages. We conduct extensive experiments to evaluate the effectiveness of our designs, and CMFAN sets new state-of-the-art performance in all of the 1-/10-/25-shot tasks of ShapeNet and ModelNet datasets.

Textbook oncological outcomes and prognosis after curative gastrectomy in advanced gastric cancer: A multicenter study.

BACKGROUND: The impact of achieving textbook oncological outcome (TOO) as a multimodal therapy quality indicator on the prognosis of advanced gastric cancer (AGC) remains inadequately assessed. METHODS: Patients with AGC who underwent curative gastrectomy between January 2010 and December 2017 at two East Asian medical centers were included. TOO was defined as achieving the textbook outcome (TO) and receiving neoadjuvant and/or adjuvant chemotherapy (NCT or ACT). Cox and logistic regression models were used to identify prognostic and non-TOO-associated risk factors. RESULTS: Among 3626 patients, 57.6% achieved TOO (TOO group), exhibiting significantly better 5-year overall survival (OS) and disease-free survival (DFS) than the non-TOO group (both p < 0.05). Multivariate Cox regression identified TOO as an independent prognostic factor for 5-year OS (HR, 0.67; 95% CI, 0.61-0.74; p < 0.001) and DFS (HR, 0.73; 95% CI, 0.66-0.81; p < 0.001). Multivariate logistic regression showed that open gastrectomy, lack of health insurance, age ≥65 years, ASA score ≥ Ⅲ, and tumor size ≥50 mm are independent risk factors for non-achievement of TOO (all p < 0.05). On a sensitivity analysis of TOO's prognostic value using varying definitions of chemotherapy parameters, a stricter definition of chemotherapy resulted in a decrease in the TOO achievement rate from 57.6 to 22.3%. However, the associated reductions in the risk of death and recurrence fluctuated within the ranges of 33-39% and 28-37%, respectively. CONCLUSIONS: TOO is a reliable and stable metric for favorable prognosis in AGC. Optimizing the surgical approach and improving health insurance status may enhance TOO achievement.

OSATS scoring confirms ICG enhancement of performance in laparoscopic radical gastrectomy: a post-hoc analysis of a randomized controlled trial.

BACKGROUND: Indocyanine green (ICG) fluorescence imaging is effective in increasing the number of lymph node dissections during laparoscopic radical gastrectomy; however, no studies have attempted to explain this phenomenon. METHODS: This study utilized the data from a previous randomized controlled trial (FUGES-012 study) investigating ICG-guided laparoscopic radical gastrectomy performed between November 2018 and July 2019. The Objective Structured Assessments of Technical Skills (OSATS) scoring system was used to grade videos from the ICG and non-ICG groups. Patients with an OSATS score greater than 29 were classified as the high-OSATS population, while those with an OSATS score less than or equal to 29 were classified as the low-OSATS population. RESULTS: A total of 258 patients were included in the modified intention-to-treat analysis: 129 in the ICG group and 129 in the non-ICG group. The OSATS score of the ICG group was higher than that of the non-ICG group (29.6±2.6 vs. 26.6±3.6; P <0.001). The ICG group underwent a significantly higher mean total number of lymph node dissections than the non-ICG group (50.5±15.9 vs. 42.0±10.3; adjusted P <0.001). The group assigned to ICG use, better OSATS (high-OSATS) scores were observed, which correlated with greater D2 lymph node retrieval (54.1±15.0 vs. 47.2±8.7; adjusted P =0.039). Finally, the ICG group had a lower rate of lymph node noncompliance than that of the non-ICG group (31.8 vs. 57.4%; P <0.001). CONCLUSIONS: By applying the ICG fluorescence navigation technique, better OSATS scores were observed, which correlated with greater lymph node retrieval and a lower lymph node noncompliance rate, as recommended for individualized laparoscopic radical gastrectomy.

A scoring model for predicting early recurrence of gastric cancer with normal preoperative tumor markers: A multicenter study.

INTRODUCTION: Prognostic factors for postoperative early recurrence (ER) of gastric cancer (GC) in patients with normal or abnormal preoperative tumor markers (pre-TMs) remain unclear. MATERIALS AND METHODS: 2875 consecutive patients with GC who underwent radical gastrectomy (RG) between January 2010 and December 2016 were enrolled and randomly divided into training and internal validation groups. ER was defined as recurrence within two years of gastrectomy. Normal pre-TMs were defined as CEA≤5 ng/mL and CA199 ≤ 37 U/mL. Least absolute shrinkage selection operator (LASSO) Cox regression analysis was used to screen ER predictors. The scoring model was validated using 546 patients from another hospital. RESULTS: A total of 3421 patients were included. Multivariate Cox analysis showed that pre-TMs was an independent prognostic factor for ER. Survival after ER was equally poor in the normal and abnormal pre-TMs groups (P = 0.160). Based on LASSO Cox regression, the ER of patients with abnormal pre-TMs was only associated with the pT and pN stages; however, in patients with normal pre-TMs, it was also associated with tumor size, perineural invasion, and prognostic nutritional index. Scoring model constructed for patients with normal pre-TMs had better predictive performance than TNM staging (concordance-index:0.826 vs. 0.807, P < 0.001) and good reproducibility in both validation sets. Moreover, through risk stratification, the scoring model could not only identify the risk of ER but also distinguish ER patterns and adjuvant chemotherapy benefit subgroups. CONCLUSION: pre-TMs is an independent prognostic factor for ER in GC after RG. The established scoring model demonstrates excellent predictive performance and clinical utility.

An international multi-institution real-world study of the optimal surveillance frequency for stage II/III gastric cancer: the more, the better?

BACKGROUND: Due to lacking evidence on surveillance for gastric cancer (GC), this study aimed to determine the optimal postsurgical surveillance strategy for pathological stage (pStage) II/III GC patients and compare its cost-effectiveness with traditional surveillance strategies. METHODS: Prospectively collected data from stage II/III GC patients ( n =1661) who underwent upfront surgery at a large-volume tertiary cancer center in China (FJMUUH cohort) between January 2010 and October 2015. For external validation, two independent cohorts were included, which were composed of 380 stage II/III GC patients at an tertiary cancer center in U.S.A (Mayo cohort) between July 1991 and July 2012 and 270 stage II/III GC patients at another tertiary cancer center in China (QUAH cohort) between May 2010 and October 2014. Random forest models were used to predict dynamic recurrence hazards and to construct individual surveillance strategies for stage II/III GC. Cost-effectiveness was assessed by the Markov model. RESULTS: The median follow-up period of the FJMUUH, the Mayo, and QUAH cohorts were 55, 158, and 70 months, respectively. In the FJMUUH cohort, the 5-year recurrence risk was higher in pStage III compared with pStage II GC patients ( P <0.001). Our novel individual surveillance strategy achieved optimal cost-effectiveness for pStage II GC patients (ICER =$490/QALY). The most intensive NCCN surveillance guideline was more cost-effective (ICER =$983/QALY) for pStage III GC patients. The external validations confirmed our results. CONCLUSION: For patients with pStage II GC, individualized risk-based surveillance outperformed the JGCTG and NCCN surveillance guidelines. However, the NCCN surveillance guideline may be more suitable for patients with pStage III GC. Even though our results are limited by the retrospective study design, the authors believe that our findings should be considered when recommending postoperative surveillance for stage II/III GC with upfront surgery in the absence of a randomized clinical trial.

Evaluation of dynamic recurrence risk for locally advanced gastric cancer in the clinical setting of adjuvant chemotherapy: a real-world study with IPTW-based conditional recurrence analysis.

BACKGROUND: The long-term dynamic recurrence hazard of locally advanced gastric cancer (LAGC) in the clinical setting of adjuvant chemotherapy (ACT) remains unclear. PURPOSE: This study aimed to investigate the dynamic recurrence risk of LAGC in patients who received ACT or not. METHODS: The study assessed data from patients with LAGC who underwent radical gastrectomy between January, 2010 and October, 2015. Inverse probability of treatment weighting (IPTW) was performed to reduce selection bias between the ACT and observational (OBS) groups. Conditional recurrence-free survival (cRFS) and restricted mean survival time (RMST) were used to assess the survival differences. RESULTS: In total, 1,661 LAGC patients were included (ACT group, n = 1,236 and OBS group, n = 425). The recurrence hazard gradually declined; in contrast, cRFS increased with RFS already accrued. Following IPTW adjustment, the cRFS rates were higher in the ACT group than those in the OBS group for patients at baseline or with accrued RFS of 1 and 2 years (p˂0.05). However, the cRFS rates of the ACT group were comparable with those of the OBS group for patients with accrued RFS of 3 or more years (p > 0.05). Likewise, the 5-year △RMST between the ACT and OBS groups demonstrated a similar trend. Moreover, the hematological metastasis rate of the ACT group was significantly lower than that of the OBS group for patients at baseline or with accrued RFS of 1 and 2 years, respectively (p˂0.05). CONCLUSIONS: Although ACT could provide substantial benefits for patients with LAGC, the differences in recurrence hazard between the ACT and OBS groups may attenuate over time, which could help guide surveillance and alleviate patients' anxiety. Further prospective large-scale studies are warranted.

Comparison of robotic versus laparoscopic versus open distal gastrectomy for locally advanced gastric cancer: a prospective trial-based economic evaluation.

IMPORTANCE: It is largely unclear whether robotic distal gastrectomy (RDG) is cost-effective for locally advanced gastric cancer (LAGC). OBJECTIVE: To evaluate the cost-effectiveness of RDG, laparoscopic distal gastrectomy (LDG), and open distal gastrectomy (ODG) for patients with LAGC. DESIGN, SETTING, AND PARTICIPANTS: Inverse probability of treatment weighting (IPTW) was used to balance baseline characteristics. A decision-analytic model was constructed to evaluate the cost-effectiveness of RDG, LDG, and ODG. EXPOSURES: RDG, LDG, and ODG. MAIN OUTCOMES AND MEASURES: Incremental cost-effectiveness ratio (ICER) and quality-adjusted life year (QALY). RESULTS: This pooled analysis of two randomized controlled trials included 449 patients: 117, 254, and 78 patients in the RDG, LDG, and ODG groups, respectively. After IPTW, RDG demonstrated its priority in terms of less blood loss, postoperative length, and complication rate (all P < 0.05). RDG also showed higher QOL with more cost, representing an ICER of $85,739.73 per QALY and $42,189.53 per QALY compared to LDG and ODG, respectively. In probabilistic sensitivity analysis, RDG achieved the best cost-effectiveness for patients with LAGC only when the willingness-to-pay threshold was > $85,739.73 per QALY, which significantly exceeded 3 times Chinese per capita GDP. Furthermore, one of the most important factors was the indirect costs of robotic surgery in terms of the cost-effectiveness of RDG compared to that of LDG or ODG. CONCLUSIONS AND RELEVANCE: Although improved short-term outcomes and QOL were seen in patients underwent RDG, the economic burden should be considered in the clinical decision-making regarding robotic surgery use for patients with LAGC. Our findings may vary in different health care settings and affordability. Trial registration CLASS-01 trial (ClinicalTrials.gov, CT01609309) and FUGES-011 trial (ClinicalTrials.gov, NCT03313700).

Semantic Consistent Embedding for Domain Adaptive Zero-Shot Learning.

Unsupervised domain adaptation has limitations when encountering label discrepancy between the source and target domains. While open-set domain adaptation approaches can address situations when the target domain has additional categories, these methods can only detect them but not further classify them. In this paper, we focus on a more challenging setting dubbed Domain Adaptive Zero-Shot Learning (DAZSL), which uses semantic embeddings of class tags as the bridge between seen and unseen classes to learn the classifier for recognizing all categories in the target domain when only the supervision of seen categories in the source domain is available. The main challenge of DAZSL is to perform knowledge transfer across categories and domain styles simultaneously. To this end, we propose a novel end-to-end learning mechanism dubbed Three-way Semantic Consistent Embedding (TSCE) to embed the source domain, target domain, and semantic space into a shared space. Specifically, TSCE learns domain-irrelevant categorical prototypes from the semantic embedding of class tags and uses them as the pivots of the shared space. The source domain features are aligned with the prototypes via their supervised information. On the other hand, the mutual information maximization mechanism is introduced to push the target domain features and prototypes towards each other. By this way, our approach can align domain differences between source and target images, as well as promote knowledge transfer towards unseen classes. Moreover, as there is no supervision in the target domain, the shared space may suffer from the catastrophic forgetting problem. Hence, we further propose a ranking-based embedding alignment mechanism to maintain the consistency between the semantic space and the shared space. Experimental results on both I2AwA and I2WebV clearly validate the effectiveness of our method. Code is available at https://github.com/tiggers23/TSCE-Domain-Adaptive-Zero-Shot-Learning.

Single-View 3D Mesh Reconstruction for Seen and Unseen Categories.

Single-view 3D object reconstruction is a fundamental and challenging computer vision task that aims at recovering 3D shapes from single-view RGB images. Most existing deep learning based reconstruction methods are trained and evaluated on the same categories, and they cannot work well when handling objects from novel categories that are not seen during training. Focusing on this issue, this paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories and encourage models to reconstruct objects literally. Specifically, we propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction. Firstly, we factorize the complicated image-to-mesh mapping into two simpler mappings, i.e., image-to-point mapping and point-to-mesh mapping, while the latter is mainly a geometric problem and less dependent on object categories. Secondly, we devise a local feature sampling strategy in 2D and 3D feature spaces to capture the local geometry shared across objects to enhance model generalization. Thirdly, apart from the traditional point-to-point supervision, we introduce a multi-view silhouette loss to supervise the surface generation process, which provides additional regularization and further relieves the overfitting problem. The experimental results show that our method significantly outperforms the existing works on the ShapeNet and Pix3D under different scenarios and various metrics, especially for novel objects.

Robust-EQA: Robust Learning for Embodied Question Answering With Noisy Labels.

Embodied question answering (EQA) is a recently emerged research field in which an agent is asked to answer the user's questions by exploring the environment and collecting visual information. Plenty of researchers turn their attention to the EQA field due to its broad potential application areas, such as in-home robots, self-driven mobile, and personal assistants. High-level visual tasks, such as EQA, are susceptible to noisy inputs, because they have complex reasoning processes. Before the profits of the EQA field can be applied to practical applications, good robustness against label noise needs to be equipped. To tackle this problem, we propose a novel label noise-robust learning algorithm for the EQA task. First, a joint training co-regularization noise-robust learning method is proposed for noisy filtering of the visual question answering (VQA) module, which trains two parallel network branches by one loss function. Then, a two-stage hierarchical robust learning algorithm is proposed to filter out noisy navigation labels in both trajectory level and action level. Finally, by taking purified labels as inputs, a joint robust learning mechanism is given to coordinate the work of the whole EQA system. Empirical results demonstrate that, under extremely noisy environments (45% of noisy labels) and low-level noisy environments (20% of noisy labels), the robustness of deep learning models trained by our algorithm is superior to the existing EQA models in noisy environments.

Unsupervised 3D Pose Transfer With Cross Consistency and Dual Reconstruction.

The goal of 3D pose transfer is to transfer the pose from the source mesh to the target mesh while preserving the identity information (e.g., face, body shape) of the target mesh. Deep learning-based methods improved the efficiency and performance of 3D pose transfer. However, most of them are trained under the supervision of the ground truth, whose availability is limited in real-world scenarios. In this work, we present X-DualNet, a simple yet effective approach that enables unsupervised 3D pose transfer. In X-DualNet, we introduce a generator G which contains correspondence learning and pose transfer modules to achieve 3D pose transfer. We learn the shape correspondence by solving an optimal transport problem without any key point annotations and generate high-quality meshes with our elastic instance normalization (ElaIN) in the pose transfer module. With G as the basic component, we propose a cross consistency learning scheme and a dual reconstruction objective to learn the pose transfer without supervision. Besides that, we also adopt an as-rigid-as-possible deformer in the training process to fine-tune the body shape of the generated results. Extensive experiments on human and animal data demonstrate that our framework can successfully achieve comparable performance as the state-of-the-art supervised approaches.

The GLIM criteria as an effective tool for survival prediction in gastric cancer patients.

BACKGROUND: The Global Leadership Initiative on Malnutrition released a new version of the malnutrition criteria (GLIM criteria). To investigate the influence of the GLIM criteria on the long-term efficacy of radical gastric cancer surgery and establish a nomogram to predict the long-term prognosis of patients with gastric cancer. METHODS: A retrospective analysis of 1121 patients with gastric cancer in our department from 2010 to 2013 was performed. A nomogram was established to predict overall survival (OS) based on the GLIM criteria. Patients were divided into the low-risk group (LRG) and high-risk group (HRG) based on the established nomogram. RESULTS: Multivariate Cox regression analyses showed that GLIM criteria was an independent risk factor for the 5-year OS (HR = 1.768, Cl:1.341-2.329, p < 0.001). The C index, AUC and Time-ROC of the nomogram were significantly better than that of GLIM criteria and traditional criteria. The 5-year OS of patients receiving adjuvant chemotherapy in the high-risk group was significantly higher than that of patients without chemotherapy (45.77% vs. 24.73%,p < 0.001). CONCLUSIONS: The GLIM criteria independently influence the long-term outcome of patients after radical gastric cancer surgery. The established nomogram can predict the long-term survival of patients with gastric cancer, and postoperative adjuvant chemotherapy for HRG can significantly improve the 5-year OS of patients.

Value of the Preoperative D-Dimer to Albumin Ratio for Survival and Recurrence Patterns in Gastric Cancer.

BACKGROUND: D-dimer (DDI) and albumin are prognostic markers for numerous cancers; however, the predictive value of the preoperative DDI-to-albumin ratio (DAR) on the survival and recurrence patterns of gastric cancer (GC) remains unclear. OBJECTIVE: The aim of this study was to explore the prognostic value of the DAR in GC. METHODS: Our study included 1766 patients with GC, divided into training and testing cohorts at a ratio of 7:3. Patients were classified into either a high-DAR group (> 0.0145) or low-DAR group (≤ 0.0145) according to the cut-off value of receiver operating characteristic (ROC) curve analysis. The relationship between the DAR and recurrence pattern was analyzed in stage II/III patients. RESULTS: Eight preoperative hematological factors were included and 17 composite inflammatory markers were constructed. ROC and random forest analyses indicated that among 17 markers, DAR was the best predictor for overall survival (OS) in GC (p < 0.01). High DAR was significantly associated with poor OS (hazard ratio [HR] 1.89, p < 0.001) and recurrence-free survival (RFS; HR 1.85, p < 0.001). Subgroup analysis showed no differences in OS and RFS between the high- and low-DAR groups in stage I or pT1/2 or pN0/1 patients; however, in stage II/III or pT3/4 or pN2/3 patients, the high-DAR group had shorter OS and RFS rates than the low-DAR group (p < 0.001). Similar results were found in the testing cohort. According to the multivariate analysis based on the training cohort, five indices, including DAR, cT stage, cN stage, age and body mass index (BMI), were incorporated to establish a nomogram model to predict the long-term prognosis of GC. The model showed comparable forecast performance in predicting OS (C-index: 0.773 vs. 0.786) and RFS (C-index: 0.788 vs. 0.795) compared with pTNM. Recurrence pattern analysis in stage II/III patients showed that the high-DAR group had a higher incidence of peritoneal implantation and early recurrence (ER) than the low-DAR group, and the post-recurrence survival in the high-DAR group was significantly shorter than that in the low-DAR group (p = 0.016). CONCLUSION: The preoperative DAR is a new biomarker for the long-term survival prediction of GC. In advanced GC, a preoperative DAR > 0.0145 aids the timely detection of ER and peritoneal recurrence after surgery, thus guiding individual follow-up strategies.

DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning.

In this work, we develop methods for few-shot image classification from a new perspective of optimal matching between image regions. We employ the Earth Mover's Distance (EMD) as a metric to compute a structural distance between dense image representations to determine image relevance. The EMD generates the optimal matching flows between structural elements that have the minimum matching cost, which is used to calculate the image distance for classification. To generate the important weights of elements in the EMD formulation, we design a cross-reference mechanism, which can effectively alleviate the adverse impact caused by the cluttered background and large intra-class appearance variations. To implement k-shot classification, we propose to learn a structured fully connected layer that can directly classify dense image representations with the EMD. Based on the implicit function theorem, the EMD can be inserted as a layer into the network for end-to-end training. Our extensive experiments validate the effectiveness of our algorithm which outperforms state-of-the-art methods by a significant margin on five widely used few-shot classification benchmarks, namely, miniImageNet, tieredImageNet, Fewshot-CIFAR100 (FC100), Caltech-UCSD Birds-200-2011 (CUB), and CIFAR-FewShot (CIFAR-FS). We also demonstrate the effectiveness of our method on the image retrieval task in our experiments.

Assessment of Laparoscopic Spleen-Preserving Hilar Lymphadenectomy for Advanced Proximal Gastric Cancer Without Invasion Into the Greater Curvature: A Randomized Clinical Trial.

Importance: The survival benefit of laparoscopic total gastrectomy combined with spleen-preserving splenic hilar lymphadenectomy (LSTG) for locally advanced proximal gastric cancer (APGC) without invasion into the greater curvature remains uncertain. Objective: To compare the long-term and short-term efficacy of LSTG (D2 + No. 10 group) and conventional laparoscopic total gastrectomy (D2 group) for patients with APGC that has not invaded the greater curvature. Design, Setting, and Participants: In this open-label, prospective randomized clinical trial, a total of 536 patients with clinical stage cT2 to 4a/N0 to 3/M0 APGC without invasion into the greater curvature were enrolled from January 2015 to October 2018. The final follow-up was on October 31, 2021. Data were analyzed from December 2021 to February 2022. Interventions: Eligible patients were randomized to the D2 + No. 10 group or the D2 group. Main Outcomes and Measures: The primary outcome was 3-year disease-free survival (DFS). The secondary outcomes were 3-year overall survival (OS) and morbidity and mortality within 30 days after surgery. Results: Of 526 included patients, 392 (74.5%) were men, and the mean (SD) age was 60.6 (9.6) years. A total of 263 patients were included in the D2 + No. 10 group, and 263 were included in the D2 group. The 3-year DFS was 70.3% (95% CI, 64.8-75.8) for the D2 + No. 10 group and 64.3% (95% CI, 58.4-70.2; P = .11) for the D2 group, and the 3-year OS in the D2 + No. 10 group was better than that in the D2 group (75.7% [95% CI, 70.6-80.8] vs 66.5% [95% CI, 60.8-72.2]; P = .02). Multivariate analysis revealed that splenic hilar lymphadenectomy was not an independent protective factor for DFS (hazard ratio [HR], 0.86; 95% CI, 0.63-1.16) or OS (HR, 0.81; 95% CI, 0.59-1.12). Stratification analysis showed that patients with advanced posterior gastric cancer in the D2 + No. 10 group had better 3-year DFS (92.9% vs 39.3%; P < .001) and OS (92.9% vs 42.9%; P < .001) than those in the D2 group. Multivariate analysis confirmed that patients with advanced posterior gastric cancer could have the survival benefit from No. 10 lymph node dissection (DFS: HR, 0.10; 95% CI, 0.02-0.46; OS: HR, 0.12; 95% CI, 0.03-0.52). Conclusions and Relevance: Although LSTG could not significantly improve the 3-year DFS of patients with APGC without invasion into the greater curvature, patients with APGC located posterior gastric wall may benefit from LSTG. Trial Registration: ClinicalTrials.gov Identifier: NCT02333721.

Learning Structural Representations for Recipe Generation and Food Retrieval.

Food is significant to human daily life. In this paper, we are interested in learning structural representations for lengthy recipes, that can benefit the recipe generation and food cross-modal retrieval tasks. Different from the common vision-language data, here the food images contain mixed ingredients and target recipes are lengthy paragraphs, where we do not have annotations on structure information. To address the above limitations, we propose a novel method to unsupervisedly learn the sentence-level tree structures for the cooking recipes. Our approach brings together several novel ideas in a systematic framework: (1) exploiting an unsupervised learning approach to obtain the sentence-level tree structure labels before training; (2) generating trees of target recipes from images with the supervision of tree structure labels learned from (1); and (3) integrating the learned tree structures into the recipe generation and food cross-modal retrieval procedure. Our proposed model can produce good-quality sentence-level tree structures and coherent recipes. We achieve the state-of-the-art recipe generation and food cross-modal retrieval performance on the benchmark Recipe1M dataset.

Depth and Video Segmentation Based Visual Attention for Embodied Question Answering.

Embodied Question Answering (EQA) is a newly defined research area where an agent is required to answer the user's questions by exploring the real-world environment. It has attracted increasing research interests due to its broad applications in personal assistants and in-home robots. Most of the existing methods perform poorly in terms of answering and navigation accuracy due to the absence of fine-level semantic information, stability to the ambiguity, and 3D spatial information of the virtual environment. To tackle these problems, we propose a depth and segmentation based visual attention mechanism for Embodied Question Answering. First, we extract local semantic features by introducing a novel high-speed video segmentation framework. Then guided by the extracted semantic features, a depth and segmentation based visual attention mechanism is proposed for the Visual Question Answering (VQA) sub-task. Further, a feature fusion strategy is designed to guide the navigator's training process without much additional computational cost. The ablation experiments show that our method effectively boosts the performance of the VQA module and navigation module, leading to 4.9 % and 5.6 % overall improvement in EQA accuracy on House3D and Matterport3D datasets respectively.

Cross-Modal Graph With Meta Concepts for Video Captioning.

Video captioning targets interpreting the complex visual contents as text descriptions, which requires the model to fully understand video scenes including objects and their interactions. Prevailing methods adopt off-the-shelf object detection networks to give object proposals and use the attention mechanism to model the relations between objects. They often miss some undefined semantic concepts of the pretrained model and fail to identify exact predicate relationships between objects. In this paper, we investigate an open research task of generating text descriptions for the given videos, and propose Cross-Modal Graph (CMG) with meta concepts for video captioning. Specifically, to cover the useful semantic concepts in video captions, we weakly learn the corresponding visual regions for text descriptions, where the associated visual regions and textual words are named cross-modal meta concepts. We further build meta concept graphs dynamically with the learned cross-modal meta concepts. We also construct holistic video-level and local frame-level video graphs with the predicted predicates to model video sequence structures. We validate the efficacy of our proposed techniques with extensive experiments and achieve state-of-the-art results on two public datasets.

A novel hematological classifier predicting chemotherapy benefit and recurrence hazard for locally advanced gastric cancer A multicenter IPTW analysis.

BACKGROUND: Effective classifiers for the prediction of individual adjuvant chemotherapy (AC) benefits are scarce. PURPOSE: This study aimed to construct a useful classifier to predict the AC benefit and recurrence hazard based on preoperative hematological indices through a multicenter database. METHODS AND RESULTS: Multivariate analysis revealing GCRF (comprehensive deep learning classifier) as an independent prognostic factor associated with overall survival (OS) and disease-free survival (DFS). Locally advanced gastric cancer (LAGC) patients are categorized into the high-risk group (HRG) and low-risk group (LRG). In HRG, OS and DFS of the AC group are significantly higher than those of the non-AC group (all p˂0.05), whereas in LRG, OS and DFS of the AC group are comparable to those of the non-AC group (all p > 0.05). Furthermore, combined GCRF with 8th AJCC TNM staging system, only 650 (51.1%) patients can benefit most from AC among 1273 patients with pStage II-III. From the perspective of recurrence pattern, the recurrence rate of HRG is significantly higher than that of LRG in any recurrence type, including local recurrence, peritoneal recurrence, and distant recurrence (all p˂0.05). Furthermore, the mean time to peritoneal recurrence and lung metastasis in HRG is earlier than that in the LRG (p = 0.028 and 0.011, respectively). CONCLUSION: In summary, our novel classifier based on deep learning preoperative hematological indices can predict not only the AC benefit of LAGC patients, but also the recurrence hazard after surgery. This classifier is expected to be an effective supplement to the 8th AJCC TNM staging system for the prediction of AC benefits and is helpful for clinical decision in AC individual administration. Further large-scale western studies are warranted.

Postoperative follow-up for gastric cancer needs to be individualized according to age, tumour recurrence pattern, and recurrence time.

BACKGROUND: Few studies have reported the association between the pattern and time point of recurrence in different groups stratified by age in postoperative survival of patients with gastric cancer. METHODS: The clinicopathological data and recurrence data of 2028 patients with GC who underwent curative surgery from January 2010 to March 2015 were enrolled in this study. Patients were grouped according to age: young group (YG) (≤45 years old) (n = 180) and non-young group (OG) (>45 years old) (n = 1848). RESULTS: A total of 2028 patients were enrolled. The young group had better 5-year OS and DFS than the non-young group. In peritoneal recurrence, the cumulative incidence of recurrence in YG was higher than that in OG (P < 0.001). In distant recurrence, the cumulative incidence of recurrence YG was always lower than that of OG (P = 0.004). Recurrence hazard function varied over time between the two groups:in the peritoneal metastasis, the recurrence hazard for YG was higher and earlier than that of OG and the YG was observed during five years after surgery with two recurrence peaks in 8.5 months and in 41.5 months. In distant recurrence, the recurrence hazard for OG had an earlier and higher single peak than that of YG (6.0 months). CONCLUSION: The recurrence characteristics of patients with gastric cancer after curative resection between young group and older group are different. Personalized follow-up strategies should be developed according to the age and time point after operation for the early detection of recurrence and making decision for further treatment.